20. BPTT Quiz 1

A folded RNN model
SOLUTION:
Equation D
Solution
\bar{z} and \bar{s} are vectors, as we indicate that they have multiple neurons in each layer. Using this logic we can understand that equations A and C are incorrect. Since w_2 connects the hidden state \bar{z} to itself, we know that we need to consider the previous timestep here. Therefore only equation D is the correct one.